The present invention relates to speech recognition systems and in particular to speech recognition systems that exploit vocal tract resonances in speech.
In human speech, a great deal of information is contained in the first three or four resonant frequencies of the speech signal. In particular, when a speaker is pronouncing a vowel, the frequencies (and to a less extent, bandwidths) of these resonances indicate which vowel is being spoken.
Such resonant frequencies and bandwidths are often referred to collectively as formants. During sonorant speech, which is typically voiced, formants can be found as spectral prominences in a frequency representation of the speech signal. However, during non-sonorant speech, the formants cannot be found directly as spectral prominences. Because of this, the term “formants” has sometimes been interpreted as only applying to sonorant portions of speech. To avoid confusion, some researchers use the phrase “vocal tract resonance” to refer to formants that occur during both sonorant and non-sonorant speech. In both cases, the resonance is related to only the oral tract portion of the vocal tract.
To detect formants, systems of the prior art analyzed the spectral content of a frame of the speech signal. Since a formant can be at any frequency, the prior art has attempted to limit the search space before identifying a most likely formant value. Under some systems of the prior art, the search space of possible formants is reduced by identifying peaks in the spectral content of the frame. Typically, this is done by using linear predictive coding (LPC) which attempts to find a polynomial that represents the spectral content of a frame of the speech signal. Each of the roots of this polynomial represents a possible resonant frequency in the signal and thus a possible formant. Thus, using LPC, the search space is reduced to those frequencies that form roots of the LPC polynomial.
In other formant tracking systems of the prior art, the search space is reduced by comparing the spectral content of the frame to a set of spectral templates in which formants have been identified by an expert. The closest “n” templates are then selected and used to calculate the formants for the frame. Thus, these systems reduce the search space to those formants associated with the closest templates.
One system of the prior art, developed by the same inventors as the present invention, used a consistent search space that was the same for each frame of an input signal. Each set of formants in the search space was mapped into a feature vector. Each of the feature vectors was then applied to a model to determine which set of formants was most likely.
This system works well but is computationally expensive because it typically utilizes Mel-Frequency Cepstral Coefficient frequency vectors, which require the application of a set of frequencies to a complex filter that is based on all of the formants in the set of formants that is being mapped followed by a windowing step and a discrete cosine transform step in order to map the formants into the feature vectors. This computation was too time-consuming to be performed at run time and thus all of the sets of formants had to be mapped before run time and the mapped feature vectors had to be stored in a large table. This is less than ideal because it requires a substantial amount of memory to store all of the mapped feature vectors.
In addition, the mapping provided by the MFCC system is difficult to invert because the formants are combined as a product before performing the windowing function.
Thus, a formant tracking system is needed that does not reduce the search space in such a way that the formants in different frames of the speech signal are identified using different formant search spaces while at the same time limiting the amount of memory and computational resources that are needed to identify the formants.
In addition, formant trackers of the past have not utilized formant targets when determining a likelihood of a change in formants over time. Instead, past systems have used generic continuity constraints. However, such systems have not performed well in non-sonorant speech regions.